Microscopy System and Method for Modifying Microscope Images in the Feature Space of a Generative Network

ABSTRACT

In a computer-implemented method for modifying microscope images, a generative model is trained using a training dataset which comprises a plurality of microscope images. After the training, the generative model is configured to compute a generated microscope image from a feature vector derived from a feature space. It is established which image properties are affected by which feature variables in the feature space. A microscope image to be modified is received and projected into the feature space in order to obtain an associated feature vector. One or more feature variables of the feature vector are modified in order to change one or more image properties, whereby a modified feature vector is generated. The modified feature vector is projected back into an image space by inputting the modified feature vector into the generative model, thereby generating a modified microscope image.

REFERENCE TO RELATED APPLICATIONS

The current application claims the benefit of German Patent ApplicationNo. 10 2021 133 868.9, filed on 20 Dec. 2021, which is herebyincorporated by reference.

FIELD OF THE DISCLOSURE

The present disclosure relates to a microscopy system and a method formodifying microscope images.

BACKGROUND OF THE DISCLOSURE

Overview images are frequently captured with modern microscopes.Overview images can be used for a sample navigation or processed(partially) automatically in order to control or monitor processes ofthe microscope. However, certain image content or image properties inparticular in this type of microscope image can hamper an automaticprocessing. For example, an overview image of a sample carrier candisplay annotations written on the sample carrier, which may causeprocessing errors in an automated processing. If a background is visiblethrough the sample carrier, an image processing program may not be ableto reliably differentiate between the background and structures on thesample carrier, e.g. a cover slip or the actual sample. Holding clipsused to hold a sample carrier on the microscope stage can also interferein an overview image. It is consequently desirable for certain imagecontent or image properties to be modified in a pre-processing of amicroscope image. Programs designed for specific applications areconceivable, for example for suppressing or removing handwrittenannotations in an image of a sample carrier. However, this makesachieving a reliably high image-processing quality for a large varietyof different image content in microscope images difficult. Providingcorresponding programs for a large variety of different image content tobe modified and updating such programs in the event of novel microscopeimages also involves considerable effort.

In general, machine-learned models are increasingly being implemented inimage processing. Reference is made to the following publications asbackground:

Karras, T., et al., “A Style-Based Generator Architecture for GenerativeAdversarial Networks” in arXiv:1812.04948v3 [cs.NE] 29 Mar. 2019: A GANwith which images of human faces are generated is described. A portraitimage is generated in which the style (e.g., pose, hairstyle, face shapeand glasses) is adopted from a predetermined image.

Karras, T., et al., “Analyzing and Improving the Image Quality ofStyleGAN” in arXiv:1912.04958v2 [cs.CV] 23 Mar. 2020: This documentdescribes a redesigned normalization of the generator of the StyleGAN inorder to avoid artefacts in generated StyleGAN images, in particularblob-like ovals in portraits or images of vehicles and animals.

Rameen, A., et al., “Image2StyleGAN: How to Embed Images Into theStyleGAN Latent Space?” in arXiv:1904.03189v2 [cs.CV] 3 Sep. 2019: Thisdocument describes a way to project a given image into the feature spaceof a ready-trained StyleGAN network.

Rameen, A., et al., “Image2StyleGAN++: How to Edit the Embedded Images?”in arXiv:1911.11544v2 [cs.CV] 7 Aug. 2020: This article describes how,for a provided image, an image that approximates the provided image canbe generated by means of the generator of a GAN. As illustrated in FIG.8 in the article, it is possible to provide a scribbled image for whichthe generator automatically reconstructs an image that appearsrelatively genuine.

Cootes, T. F., et al., “Active Appearance Models” in IEEE TRANSACTIONSON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 23, NO. 6, JUNE 2001:This document describes a parameterized model by means of which aprovided image, e.g. a photo of a human face, can be reconstructed.

SUMMARY OF THE DISCLOSURE

It can be considered an object of the invention to provide a microscopysystem and a method which enable a quality enhancement of capturedmicroscope images in a flexible manner. The quality enhancement canrelate to, for example, a suppression of interfering image artefacts orirrelevant elements or to an enhancement of the visibility of relevantstructures.

This object is achieved by means of the methods and the microscopysystem with the features of the independent claims.

A method according to the invention for modifying microscope imagescomprises at least the following steps:

-   -   A generative model is trained using a training dataset which        comprises a plurality of microscope images so that, after the        training, the generative model is configured to calculate a        generated microscope image from a feature vector derived from a        feature space.    -   It is established which image properties are affected by which        feature variables in the feature space.    -   A microscope image to be modified is received.    -   The microscope image to be modified is projected into the        feature space in order to obtain an associated feature vector.    -   One or more image properties are changed by modifying one or        more feature variables of the feature vector, whereby a modified        feature vector is generated.    -   The modified feature vector is projected back into an image        space by inputting the modified feature vector into the        generative model, thereby generating a modified microscope        image.

Thus, a generative model is initially learned which can generatemicroscope images that appear to come from a statistical distribution ofpredetermined microscope images of the training dataset. Generatedmicroscope images thus closely resemble the predetermined microscopeimages of the training data in terms of their type so that it may not bepossible to distinguish whether a microscope image is generated orgenuine. An image content or an image property of a microscope image tobe modified is then changed, not by changing the microscope imageitself, but by changing a feature vector corresponding to the microscopeimage to be modified. The modified microscope image is then calculatedfrom the modified feature vector.

In a further embodiment of the invention, a ready-trained generativemodel is used so that it is not necessary to carry out the steps of thetraining as part of the claimed method. Such a method according to theinvention for modifying microscope images comprises at least thefollowing steps:

-   -   A microscope image to be modified is received.    -   A modified microscope image is generated by means of a        generative model, wherein the generative model is trained, using        a training dataset which comprises a plurality of microscope        images, to calculate a generated microscope image from a feature        vector derived from a feature space. A relationship between one        or more image properties and a respective feature variable in        the feature space is recorded. To generate the modified        microscope image by means of the generative model, at least the        following steps are performed:    -   The microscope image to be modified is projected into the        feature space in order to obtain an associated feature vector.    -   One or more feature variables of the feature vector are modified        in order to change one or more image properties, whereby a        modified feature vector is generated.    -   The modified feature vector is projected back into an image        space by inputting the modified feature vector into the        generative model, thereby generating a modified microscope        image.

The invention makes it possible to change image properties, inparticular image content, in a relatively simple manner so as to achieveresults that appear genuine. It is thus possible to, inter alia,compensate for unfavorable imaging conditions, remove potentiallyinterfering structures on sample carriers from the microscope image,highlight image structures to be analyzed, suppress a background, orreduce image noise. The thus modified microscope image can be easier tointerpret for an observer and/or yield more reliable results in asubsequent image processing.

The invention also relates to a microscopy system which comprises amicroscope for capturing microscope images and a computing device whichis configured to carry out the method according to the invention.

The invention additionally relates to a computer-readable storage mediumcomprising commands which, when executed by a computer, cause thecomputer to execute the method according to the invention.

Optional Embodiments

Variants of the microscopy system according to the invention and of themethod according to the invention are the object of the dependent claimsand are explained in the following description.

Generative Model

A generative model can generally be understood as a model or neuralnetwork which has been adapted to be able to generate from an input, inparticular from a random input, images that appear to come from astatistical distribution of predetermined microscope images of atraining dataset. Generated images thus correspond to the microscopeimages of the training dataset in terms of their type.

The generative model can be formed by a generator of a generativeadversarial network (GAN). A GAN comprises two networks, namely agenerator and a discriminator, which are trained together using thetraining dataset. The generator generates an output image from anentered (random) vector. The discriminator receives the microscopeimages of the training dataset and the generated output images of thegenerator as input. The discriminator is intended to establish whetheran input image is a genuine image, i.e. a microscope image of thetraining dataset, or a generated output image. A loss function to beminimized or a reward function to be maximized is defined accordingly.Conversely, the generator is intended to be able to generate outputimages for which the discriminator is unable to assess correctly whetherthey originate in the training dataset. The loss/reward function of thegenerator thus results from the loss/reward function of thediscriminator and both are trained at the same time, i.e. in alternatingsteps. Upon completion of the training, the generator is able togenerate, from a random vector, an image that corresponds in type andcontent to the distribution of the microscope images of the trainingdata. It is also possible to use a StyleGAN as explained in greaterdetail later on. The terms “generative adversarial network” and“generative adversarial networks” in the singular and plural areintended to be understood as fundamentally synonymous in the presentdisclosure.

The generative model can alternatively be formed by a decoder of anautoencoder. An autoencoder comprises an encoder and a subsequentdecoder. In the training, the microscope images of the training datasetare input into the encoder. From each microscope image, the encodercalculates a feature vector, which is a compressed representation of theinput image. A space formed by all possible feature vectors is called alatent space or feature space. The feature vector generated by theencoder is input into the decoder, which calculates an output imagetherefrom. A loss function measures differences between the output imageand the associated microscope image. Through minimization of the lossfunction, the autoencoder is trained to be able to generate outputimages that resemble the microscope images of the training dataset. Asdescribed in greater detail later on, the decoder can be implementedafter the training separately as a generative model of embodiments ofthe invention.

An autoencoder can in particular take the form of a variationalautoencoder. In this case, the encoder does not generate a point in thelatent space but rather a distribution. The distribution can be definedby a (centre) point and a width. An input of the decoder is now randomlydrawn from the distribution provided by the encoder. This reinforcesthat points lying close together in the latent space to result insimilar generated images. An order or structure of the latent space isthus increased.

Alternatively, it is also possible for the generative model to belearned by principal component analysis (PCA) using the trainingdataset. A plurality of microscope images of the training dataset can beidentical except for one image property; for example, the microscopeimages can differ alone in a brightness or in reflections on the sampleor sample carrier. By means of PCA, it is possible to establish aparameter which modifies brightness and otherwise leaves a microscopeimage unchanged. It is also possible to use PCA to establish a furtherparameter by means of which reflections can be amplified or reduced.

An active appearance model can also act as a generative model. In anactive appearance model, a shape model determines the position ofprominent points and thus takes into account the shape variations ofstructures depicted in the microscope images of the training dataset.Prominent points can be predetermined in the form of annotations for thetraining dataset. A texture model determines texture variations, i.e.different pixel values, after the microscope images have been broughtinto a uniform shape by the shape model. The models can likewise beestablished by PCA.

An output of a generative model is or comprises an image. Forconciseness, different embodiments are described in the presentdisclosure in which a single image is output. Generally speaking, thisis intended to be understood in the sense of “at least one” image sothat, depending on its design, the generative model is also capable ofoutputting a plurality of images or three-dimensional/volumetric imagedata from an input.

Feature Vector and Feature Space

Inputs/input data entered into the ready-trained generative model can beunderstood as feature vectors. The feature vector can in principle haveany dimension, i.e. be formed by in principle any number of parameters,which can in particular be independent of one another and which can becalled feature variables. The feature vector defines a point in a spacecalled feature space or latent space. The generative model generates amapping of a point of the latent space onto an output image, i.e. onto agenerated microscope image. The generative model is thus able togenerate a microscope image that looks genuine from a (in particularrandom) feature vector. A feature vector can also be described as arepresentation of an associated microscope image in the feature spaceand is also referred to as a latent space representation, latent code orlatent vector.

An input can be fed to the generative model at one or more differentpoints. An input thus does not necessarily have to be fed (exclusively)to a first layer of the generative model, but can alternatively oradditionally be fed to one or more other layers, as in the case of aStyleGAN architecture. In the case of an input at a plurality of pointsin the generative model, the same vector or different vectors can beinput at the plurality of points. A feature vector in the sense of thisapplication can represent the input data or input vectors collectively.

It is possible for a further neural network to be implemented in thetraining upstream of the generative model. If the generative model isthe generator of a StyleGAN, a mapping network, for example, is used ina first step. The mapping network can comprise, e.g., a plurality offully connected layers and generates from entered data an output whichis input into the generator. The mapping network thus performs a mappingof input data, i.e. a mapping of a random vector/feature vector z from afeature space Z, to a (feature) vector w in another feature space W. Thefeature space W can be better adapted to the training image datacompared to the feature space Z so that feature variables or axes of thefeature space W are better separated from one another than the axes ofthe feature space Z in terms of the image properties they encode.Although it is in principle possible for the modification of a featurevector described in greater detail later on to occur in the featurespace Z, a modification in the feature space W can be preferrable.

Semantics of Feature Variables of the Feature Space

A proximity between points in the feature space corresponds to asimilarity of the associated generated microscope images. A direction inthe feature space typically determines an image property. It is thuspossible to establish, through investigation of the feature space, howimage properties relate to directions in the feature space. Axes of thefeature space can be orthogonal to one another and can be called featurevariables. The entries of a feature vector are accordingly values of thevarious feature variables that span the feature space and are alsoreferred to as parameters of the feature vector in the presentdisclosure. If a feature variable is changed for a given feature vector,it is possible to observe an effect in the generated microscope image.It can be established in this manner which image property relates to thefeature variable. This can be carried out for each feature variable oraxis of the feature space in order to identify different modifiableimage properties. This way, a user or computer program cansystematically change components of feature vectors and observe whichimage property changes with which component in order to establish acorrelation between feature variables and image properties.

To investigate the feature space, it is also possible to respectivelyuse two microscope images which differ alone in one image property ofinterest: The difference between their representations in the featurespace is a vector, which corresponds to a feature variable and which canexhibit essentially any orientation in the feature space. This featurevariable thus describes the difference between the two microscope imageswith respect to the image property. For example, the two microscopeimages can differ alone in a contamination of the sample carrier. It isthereby possible to establish a feature variable that affects a samplecarrier contamination in the generated microscope image. It is alsopossible for more than two microscope images to be classified into twogroups according to an image property of interest. For example,microscope images which show different sample carrier types can beclassified into one of the two groups as a function of their samplecarrier contamination. The feature vectors are then averaged for allmicroscope images of the same group, i.e. a centroid is calculated fromthe points in the latent space. A difference or vector between the twocentroids of the two groups defines a feature variable, which in theaforementioned example encodes the image property “sample carriercontamination”. Generally speaking, it is also possible to form, insteadof two groups, a plurality of ordinal groups into which the microscopeimages are sorted, for example into the four groups: Sample carriercontamination: very low/low/high/very high. The feature vectors ofmicroscope images of the same group are then averaged to form acentroid. A difference between the centroids of two consecutive groups(e.g., the groups “sample carrier contamination very low” and “samplecarrier contamination low”) now forms a feature variable between thesegroups.

In order to determine a relationship regarding an image property (e.g.reflections present: yes or no), it is also possible to establish ahyperplane in the feature space. A hyperplane is a multi-dimensionalplane whose dimension is 1 less than the dimension of the feature space.A plurality of microscope images are divided into two groups accordingto an image property of interest, for example whether or not interferingreflections are visible in the microscope image. A respectiverepresentation (a point) in the latent space is established for themicroscope images. A hyperplane is then established which separates thepoints of the two groups as accurately as possible. A vectorperpendicular to the hyperplane indicates a feature variable: In thecited example, a given feature vector, i.e. a point in the featurespace, can be shifted according to the vector established in thedescribed manner in order to amplify or attenuate an interferingreflection in the associated microscope image.

A semantics of feature variables can also be established by means of aclassification or regression model. A prerequisite here is a pluralityof images with associated feature vectors. The images are classifiedmanually or by means of a program with respect to one or more imageproperties and one or more corresponding annotations are assigned, e.g.“contamination on the sample carrier low/medium/high”. A classificationor regression model is now trained to calculate from the featurevariables predictions to match the annotations. Thus learned functionsof the model can subsequently be used in the feature space as transitiondirections which describe the associated image property. Thisestablishes, e.g., a direction in the feature space that affects theimage property “contamination on the sample carrier” between the classes“low/medium/high”.

Projecting the Microscope Image to be Modified into the Feature Space

A microscope image to be modified is not changed directly; rather, in afirst step, a feature vector corresponding to a generated microscopeimage that is as consistent as possible with the microscope image to bemodified is established. In other words, a feature vector is soughtwhich, when input into the generative model, yields a generatedmicroscope image that is ideally identical to the microscope image to bemodified. Finding this feature vector can be referred to as projectingor “embedding” the microscope image to be modified into or in thefeature space. Different options exist for this calculation depending onthe design of the generative model.

For example, it is possible to start with an initial feature vector w.An optimized feature vector w* is now sought which optimizes a lossfunction measuring the similarity between a predetermined image (i.e.the microscope image to be modified) and the output image calculated bythe generative model from the input feature vector. The initial featurevector w can consist of, e.g., random values or be predetermined in someway. The iterative adjustment for calculating the optimized featurevector w* can be calculated via a gradient descent method, which is alsocalled projected gradient descent (PGD). A similarity between apredetermined image and an output image can be measured for each pixeldirectly with such an image pair. Alternatively, it is also possible tomeasure, for example, a perceptual loss to which end a pre-trainedfeature extractor, e.g. a VGG network, is used in order to respectivelycalculate an output (abstract features) from the predetermined image andfrom the output image, and the distance or difference between theabstract features is minimized by adjusting the feature vector w in thelatent space.

Alternatively, an encoder can be learned which calculates a projectionof a predetermined image onto a representation in the latent space, i.e.a correlation between the predetermined image and a representation inthe latent space. Such an encoder can be, e.g., the encoder of avariational autoencoder, which is particularly suitable if the decoderof the variational autoencoder is used as the generative model. If, onthe other hand, the generative model is learned by means of a GANarchitecture, a separate encoder can be learned upon completion of thetraining of the GAN.

For example, this encoder can be trained using pairs of predeterminedmicroscope images and associated feature vectors in a supervisedlearning process to calculate, from the predetermined microscope images,feature vectors which ideally match the predetermined feature vectors.

Alternatively, the encoder can be trained to calculate feature vectorsfrom predetermined microscope images by inputting the output of theencoder into the ready-trained generative model. The ready-trainedgenerative model calculates, from the outputs of the encoder, generatedmicroscope images whose correspondence with the predetermined microscopeimages is measured by a loss function to be optimized. Alternatively, aGAN can be supplemented by an encoder so that the encoder is trainedtogether with the generator and the discriminator. Upon completion ofthe training, the encoder can be used directly in the inference phase inorder to calculate an associated latent code/feature vector from amicroscope image to be modified.

If a reversible generative model is used as the generative model, anassociated feature vector in the latent space can be calculated directlyfrom a microscope image to be modified by means of the reversiblegenerative model without the need for the approximation techniquesdescribed in the foregoing.

Image Properties

The one or more image properties that can be modified by a change in thefeature vector can relate to one or more of the following:

-   -   an exposure of an image or image area, e.g. of an object in the        image, such as an adhesive label, sample carrier or cover slip        area, wherein a modification of the feature vector is in        particular able to compensate an overexposure or underexposure;    -   a contamination of a sample carrier or cover slip area, wherein        a modification of the feature vector can change, in particular        reduce, a degree of contamination visible in the modified        microscope image in the image area of the sample carrier or        cover slip;    -   a contrast of cover slip edges, wherein a modification of the        feature vector can change a visibility of cover slip edges, in        particular a difference in brightness between cover slip edges        and adjacent image areas;    -   reflections, local dimming or other artefacts on a sample        carrier, wherein a modification of the feature vector affects an        intensity of reflections, local dimming or other artefacts;    -   background artefacts visible through a transparent sample        carrier, for example LEDs or other light sources of an overview        illumination unit, wherein the background artefacts can be        attenuated or removed via a modification of the feature vector;    -   a background illumination or brightness; in particular, an image        brightness in image areas outside the sample carrier can be        changed via a modification of the feature vector;    -   labelling on a sample carrier; for example, a visibility of        labelling can be reduced via a modification of the feature        vector.

Each of the cited image properties can be defined or influenced by arespective feature variable. A feature vector comprises all featurevariables so that an image property can be changed in a targeted mannerby changing the corresponding feature variable of the feature vector.

Modifying the Feature Vector

A modification of the feature vector for a microscope image is carriedout by changing one or more of the parameters (feature variables) of thefeature vector.

To this end, it is possible to provide a selection option via which auser can specify an intended change in the at least one image property.One or more feature variables of the feature vector are modified inaccordance with the intended change. For example, a slider or numberinput field can be provided on a computer screen together with adesignation of the associated image property so that, for example, it ispossible to change the image property “contamination of the samplecarrier” via a slider. The modified microscope image calculated with theimage property change currently specified by the user can optionally bedisplayed together with the selection option. The effect of a change canthus be viewed directly so that the user can establish a suitablemodification. Optionally, the microscope image to be modified is alsodisplayed together with the selection option and the modified microscopeimage.

Alternatively or additionally, a modification of a feature variable ofthe feature vector can occur automatically or be proposed according topredetermined criteria. An automatically proposed modification can be,for example, the start value of a feature variable which cansubsequently be changed manually by means of the described slider.Alternatively, an automatic modification can also be carried out withoutany user interaction. An automatic modification can occur, for example,by means of a threshold value comparison, wherein a feature variable ischanged in the direction of a predetermined ideal value as a function ofthe threshold value comparison. For example, if a feature variabledescribes the image property “contrast of cover slip edges”, a thresholdvalue for a minimum contrast can be stored in the form of a minimumvalue of this feature variable. If the minimum contrast is not reached,the feature variable is automatically changed to the threshold value orto a higher value so as to attain the minimum contrast in the modifiedmicroscope image.

The thus modified feature vector is input into the generative model,which calculates a generated microscope image therefrom, which is calledthe modified microscope image.

Checking the Modified Microscope Image

An automatic check of the modified microscope image can be carried outin order to reduce the likelihood that any image errors caused by themodification go unnoticed. The modified microscope image can be inputinto a trained inspection model to this end. The latter can be trainedto establish whether image artefacts were caused by the modification ofthe feature vector. An output of the inspection model can thus be aconfidence estimate regarding whether the modified microscope image istrustworthy. The inspection model can take the form of, e.g., an anomalydetector and be learned by means of an unsupervised learning processusing training data comprising exclusively (modified or unmodified)microscope images that have been classified as correct or trustworthy bya user.

The inspection model can also be formed by the discriminator of agenerative adversarial network (GAN). If the generative model is formedby a generator of a GAN, the discriminator of this GAN can beimplemented as the inspection model. The modified microscope image isinput into the discriminator, which outputs an estimate as to whetherthe modified microscope image is genuine or generated. If thediscriminator assumes a genuine microscope image, it can be assumed thatno image artefacts were created by the modification of the featurevector. The discriminator can be designed as a regression model andaccordingly output a free value within a number range, wherein the upperand lower limits of the number range respectively represent a reliableclassification into a genuine or replicated microscope image. In orderto form the inspection model, a threshold value between theseclassifications can generally be defined arbitrarily.

Subsequent Image Utilization

The modified microscope image can subsequently be used or furtherprocessed in a workflow of the microscope. For example, the modifiedmicroscope image can act as a navigation map or serve to form anavigation map on which a user can select a location which can then beautomatically positioned or analyzed by the microscope. For example, amotorized sample stage can be automatically adjusted so that a selectedlocation lies on an optical axis of an objective in use or in the imagecentre of images to be captured.

The modified microscope image can also be input into an image processingprogram, which calculates an image processing result for an input image.The image processing program can be, e.g., a machine-learnedsegmentation model, detection model, classification model or a model forimage-to-image mapping. An image-to-image mapping can effect, forexample, a virtual staining, a noise suppression or a resolutionenhancement. Segmentation, detection or classification can be used toestablish, for example, the presence, a type and/or a position ofcertain components, for example of a sample carrier, cover slip orsample. The method according to the invention is carried out in a firststep, whereby a modified microscope image is calculated. The latter isthen entered into the image processing program. The modification of oneor more feature variables of the feature vector can occur in accordancewith requirements of the image processing program (automatically). Forexample, a requirement can be that limits stipulated for the imageprocessing program pertaining to image contrast, image noise or samplecarrier contamination must be observed, as otherwise the imageprocessing program may not function reliably. If the feature variablesof a microscope image to be modified indicate that the stipulated limitsare not observed, the feature variables are modified accordingly. Thethus calculated modified microscope image is then input into the imageprocessing program, which calculates the image processing resulttherefrom.

Microscope Images

A microscope image can be understood as an image captured by amicroscope or calculated using measurement data of a microscope. Inparticular, the microscope image can be formed by one or more raw imagesor by already processed images of the microscope. The microscope imagecan also be an overview image of an overview camera on the microscope orhave been calculated from measurement data of at least one overviewcamera. If the microscope in question is a light microscope, themicroscope image can also be a sample image captured by a sample camerawhich is provided in addition to the overview camera and which capturesan image with a higher magnification than the overview camera. It isalso possible for microscope images to have been generated by othertypes of microscopes, for example by electron microscopes or atomicforce microscopes.

The described modification of the feature vector is particularlysuitable for overview images. If there should occur undesired changes inthe image content of the actual sample as a result of the modification,such changes are less damaging with overview images than they are withsample images. For example, if the overview image is used for samplenavigation or to identify a sample carrier being used, undesired changesin the image content of the sample typically do not have any negativeconsequences.

General Features

A microscopy system denotes an apparatus which comprises at least onecomputing device and a microscope. A microscope can in particular beunderstood as a light microscope, an X-ray microscope, an electronmicroscope or a macroscope.

The computing device can be designed in a decentralized manner, bephysically part of the microscope or be arranged separately in thevicinity of the microscope or at a location at any distance from themicroscope. It can generally be formed by any combination of electronicsand software and can comprise in particular a computer, a server, acloud-based computing system or one or more microprocessors or graphicsprocessors. The computing device can also be configured to controlmicroscope components.

Method variants can optionally comprise the capture of at least onemicroscope image by the microscope while in other method variants anexisting microscope image is loaded from a memory.

Descriptions in the singular are intended to cover the variants “exactly1” as well as “at least one”. Descriptions according to which amicroscope image is input into one of the described models are intendedto comprise, for example, the possibilities that exactly one or at leastone microscope image is used. A common processing of a plurality ofmicroscope images can be suitable, e.g., when the microscope images forman image stack (z-stack) showing sample layers of a same sample atdifferent depths or are images of the same sample captured insuccession. Volumetric image data is also intended to be understood inthe context of the present disclosure as “a plurality of microscopeimages” so that it is also possible to establish and modify a featurevector pertaining to volumetric image data.

A generative model and other learned models described herein can belearned by a learning algorithm using training data. The models canrespectively comprise, for example, one or more convolutional neuralnetworks (CNNs), which receive a vector, at least one image or imagedata as input. A learning algorithm uses the training data to definemodel parameters of the model. A predetermined objective function can beoptimized to this end, e.g. a loss function can be minimized. The modelparameter values are modified to minimize the loss function, which canbe calculated, e.g., by gradient descent and backpropagation. In thecase of a CNN, the model parameters can in particular comprise entriesof convolution matrices of the different layers of the CNN. Layers thatdo not follow each other directly can optionally be connected byso-called “skip connections”, whereby the output of a layer is passed onnot only to the immediately following layer but additionally to anotherlayer. Other deep neural network model architectures are also possible.A space of possible outputs of the generative network is called an imagespace.

The characteristics of the invention that have been described asadditional apparatus features also yield, when implemented as intended,variants of the method according to the invention. Conversely, amicroscopy system or in particular the computing device can also beconfigured to carry out the described method variants. While aready-trained model is used in some variants, other variants of theinvention result from the implementation of the corresponding trainingsteps, and vice versa.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the invention and various other features andadvantages of the present invention will become readily apparent by thefollowing description in connection with the schematic drawings, whichare shown by way of example only, and not limitation, wherein likereference numerals may refer to alike or substantially alike components:

FIG. 1 schematically shows an example embodiment of a microscopy systemof the invention;

FIG. 2 schematically shows a microscope image to be modified;

FIG. 3 schematically shows the structure and a training of a GANaccording to example embodiments of methods of the invention;

FIG. 4 schematically shows training images;

FIG. 5 schematically shows further training images;

FIG. 6 schematically shows a feature space, feature vectors and featurevariables;

FIG. 7 schematically indicates a semantics of feature variables;

FIG. 8 schematically shows a further feature variable;

FIG. 9 schematically shows an example embodiment of a method of theinvention;

FIG. 10 schematically shows the structure and the training of avariational autoencoder according to example embodiments of methods ofthe invention; and

FIG. 11 shows flowcharts relating to example embodiments of methods ofthe invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Different example embodiments are described in the following withreference to the figures.

FIG. 1

FIG. 1 shows an example embodiment of a microscopy system 100 accordingto the invention. The microscopy system 100 comprises a computing device10 and a microscope 1, which is a light microscope in the illustratedexample, but which in principle can be any type of microscope. Themicroscope 1 comprises a stand 2 via which further microscope componentsare supported. The latter can in particular include: an illuminationdevice 5; an objective changer/revolver 3, on which an objective 4 ismounted in the illustrated example; a sample stage 6 with a holdingframe for holding a sample carrier 7; and a microscope camera 8. Whenthe objective 4 is pivoted into the light path of the microscope, themicroscope camera 8 receives detection light from a sample area in whicha sample can be located in order to capture a microscope image. A samplecan be any object, fluid or structure. The microscope 1 optionallycomprises an additional overview camera 9 for capturing an overviewimage of a sample environment. A field of view 9A of the overview camera9 is larger than a field of view when a sample image is captured. In theillustrated example, the overview camera 9 views the sample carrier 7via a mirror 9B. The mirror 9B is arranged on the objective revolver 3and can be selected instead of the objective 4. In variants of thisembodiment, the mirror is omitted or a different arrangement of themirror or some other deflecting element is provided. The computingdevice 10 comprises a computer program 11 stored on a data carrier forprocessing a microscope image according to a method according to theinvention. This is discussed in greater detail with reference to thefollowing figures.

FIG. 2

FIG. 2 schematically shows a microscope image 20, which in this exampleis an overview image of an overview camera. In principle, however, asample image captured using a microscope objective can also serve as amicroscope image.

A sample carrier 7 with a cover slip is discernible in the microscopeimage 20. Cover slip edges 17 appear as a bright frame. Clips of aholding frame 16 appear as dark shadows and impair the image quality.Should the microscope image 20 be used, for example, as a navigationmap, the depiction of the holding frame 16 may irritate a user.Moreover, the darker image areas of the holding frame 16 can impair apotential subsequent processing, for example a segmentation or anautomatic detection and positioning of an area within the cover slipedges 17.

Image processing is intended to improve an image quality of themicroscope image 20 and in particular to remove artefacts such as theshadows of the holding frame 16. However, a corresponding modificationis not carried out in the microscope image 20 itself, but rather in arepresentation of the microscope image 20 in the form of a featurevector from which a generative model can calculate an image. This isdescribed in greater detail in the following.

FIGS. 3 to 5

FIG. 3 schematically shows the structure and the training of generativeadversarial networks GAN comprising a generator G and a discriminator D.In the illustrated example, a network called a mapping network MN isalso used. This can be considered part of the GAN or as a separatenetwork upstream of the GAN. The generator G is also referred to in thepresent disclosure as generative network Gen.

The generator G is intended to be able to generate generated images(microscope images) 40 that are indistinguishable from predeterminedtraining images T, which can be captured microscope images.Indistinguishable can be understood in the sense that generated images40 appear to come from the same distribution as the training images T sothat the discriminator D is not able to distinguish generated images 40from training images T.

A training dataset or training images T are shown by way of example inFIGS. 4-5 . The training images T comprise microscope images 20A-20G,which should be representative of the microscope images that will beprocessed after the training. The microscope images 20A-20G are overviewimages of sample carriers and differ with respect to a plurality ofimage properties, in particular with respect to a visibility of a coverslip and cover slip edges 17, contaminants on the sample carrier 7 (e.g.dust particles, lint or fingerprints), a visibility of a labelling field18, image brightness and contrast and darkened areas caused by holdingclips 16 for the sample carrier 7.

A training of the GAN using the training images T is described in thefollowing with reference to FIG. 3 . A starting vector, e.g. a randomvector z chosen from a space Z, is input into a network called a mappingnetwork MN. The random vector z comprises a plurality of variables thatare independent of one another, for example 512 variables, whose valuescan be chosen randomly. The mapping network MN can comprise, e.g., aplurality of fully connected layers through which the starting or randomvector z is mapped to a vector w. The vector w can have a dimensiondifferent from the random vector z or the same dimension as the randomvector z, i.e. consist of, e.g., 512 variables. In general, the vector wdetermines a point in a space W, which is also called the feature spaceW.

The vector w is input into the generator G, which can comprise, interalia, a plurality of convolutional layers. An output of the generator Gin this example is a two-dimensional image, which is called a generatedimage or generated microscope image 40 in the present disclosure.

Either a (genuine) microscope image 20A-20G of the training data T or agenerated microscope image 40 of the generator G is input into thediscriminator D. An output of the discriminator D should be adiscrimination result d that indicates whether the discriminator Dclassifies an input image as a genuine microscope image 20A or as agenerated microscope image 40. The discrimination result d is enteredinto a loss function L. In order to adjust model parameter values(weights) of the generator G, the loss calculated by the loss function Lis run through the layers of the discriminator D and subsequentlythrough the layers of the generator G by means of backpropagation,wherein gradients for modifying the respective model parameter valuesare obtained for each layer. In order to adapt the generator G in atraining step, typically alone the model parameter values of thegenerator G are modified, e.g. entries of its convolution matrices,while the discriminator D remains unchanged. The mapping network MN canbe trained together with the generator G, in particular likewise via theloss function and backpropagation implemented for the generator G. In atraining step for the discriminator D, the loss calculated by the lossfunction L by means of backpropagation is used to adjust model parametervalues of the discriminator D. It is possible to use different lossfunctions in the training, namely a generator loss function and adiscriminator loss function. These can be derived from the same lossfunction L, e.g., by omitting in the training of the generator the partsof the loss function L which relate to discrimination results d forinput genuine microscope images 20A-20G. The GAN can also be designed asa Wasserstein GAN, in which a modified (vis-à-vis classic GANs) lossfunction L is used.

In the training of the generator G, the model parameter values of thelatter are modified so that the discriminator D is ideally unable todistinguish generated microscope images 40 from genuine microscopeimages 20A-20G.

Upon completion of the training, the generator G is able to generatemicroscope images that appear genuine from different vectors in thespace Z from which the random vector z derives or from different vectorsin the space W from which the feature vector w derives. By means of thetraining of the generator G, the spaces Z and W obtain a structure, i.e.points or vectors close together in the space Z or in the space W resultin similar microscope images, while points that are more distant fromone another result in very different microscope images. The spaces(feature spaces) Z and W are spanned by a plurality of axes or featurevariables that affect a microscope image generated by the generator G indifferent ways.

For a ready-trained generator G with an optional mapping network MN, thefeature space Z or W is investigated, i.e. it is established whateffects changes in a feature variable of a vector in the space Z or of avector in the space W have on the microscope image generated from thesame.

To better elucidate these aspects, the feature space W and (feature)vectors in this space are described in the following with reference toFIG. 6 . The following description applies analogously to the featurespace Z, which could also be adduced in the following.

FIG. 6

FIG. 6 illustrates the feature space W in which a feature vector w1 isplotted. The feature space W is also referred to as the latent space andthe feature vector w1 is also called the latent code of a microscopeimage. The feature space W is spanned by a plurality of axes or featurevariables, of which three feature variables a, b and u are shown by wayof example. The number of feature variables can be, for example, 512.The feature vector w1 is formed by numerical values a1, b1, u1 for thefeature variables. If the feature vector w1 is input into the generatorG described in the foregoing, the generator G can calculate a specificmicroscope image therefrom.

To investigate the feature space W, it is possible to change, forexample, the feature variable b and observe the effect on the microscopeimage generated in the process. In the example shown, the numericalvalue b1 of the feature vector w1 is changed to a numerical value b2,whereby a changed feature vector w2 is generated. By comparing the twomicroscope images calculated from the feature vectors w1 and w2, a usercan determine which image property is changed by the feature variable b.This procedure can be carried out for all feature variables a, b, . . ., u in a plurality of feature vectors.

Other approaches are also possible for establishing a semantics of thefeature variables, i.e. for determining a correlation between imageproperties and feature variables. To avoid redundancy here, reference ismade to the foregoing general description of the present application.

FIGS. 7 and 8

FIG. 7 shows an established correlation of feature variables a-u of thefeature space W with image features B. The correlation is valid for aspecific, ready-trained generative network Gen. If the generativenetwork Gen were to be re-trained (with the same or different trainingdata), the feature space W would acquire a different structure so thatthe correlation between feature variables a to u and image features Bwould have to be re-established.

In the example shown, the feature variable a determines a color of anadhesive label on a sample carrier. The feature variable b determines avisibility of the clips of a holding frame in a microscope image. Thefeature variable c determines a contamination on the sample carrier,e.g. punctiform grains of dust or lint. The feature variable ddetermines the illumination, in particular a total image brightness. Thefeature variable e determines a visibility or suppression of abackground visible through a transparent sample carrier and/or laterallynext to the sample carrier. The feature variable u determines a positionof the holding frame within the microscope image.

A feature variable does not necessarily have to be one of the axes thatspan the feature space. Rather, a feature variable can have an inprinciple arbitrary direction within the feature space W. This isillustrated in FIG. 8 , which shows a feature variable F formed as alinear combination of the feature variables a to u which span thefeature space. The feature variables a to u are multiplied by specificfactors F₁, F₂, F₃ in order to form the feature variable F. In thisexample, the feature variable F indicates the image property “visibilityof cover slip edges”. The feature variable F and its factors F₁-F₃ canbe established by comparing the feature vectors of a plurality ofmicroscope images. For example, the feature variable F can beestablished by starting with a plurality of microscope images whichdiffer in one image property. The corresponding feature vectors in thefeature space W are established for these microscope images. Adifference between the feature vectors reveals the feature variablewhich primarily affects the cited image property and does not affect orhardly affects other image properties. This feature variable can be avector with an essentially arbitrary orientation in the feature space W.The microscope images employed for this purpose do not all have to beunedited microscope images as captured by the microscope of FIG. 1 .Rather, it is also possible for a microscope image to have been modifiedmanually or by means of a separate image processing program, e.g. inorder to improve visibility of the cover slip edges (by increasing adifference in brightness between the cover slip edges and thesurrounding area in the microscope image) or by removing contaminantsand artefacts on the sample carrier from a microscope image throughimage processing. Such modified microscope images can also form part ofthe training dataset of the generative model.

An image property B of a particular microscope image can now be changedin a targeted manner as described in greater detail with reference tothe following figure.

FIG. 9

FIG. 9 illustrates the editing of a microscope image 20 to be modifiedin the feature space.

It is not necessary for the microscope image 20 to be modified to havebeen part of the training dataset T of the described generative model G.In the microscope image 20 to be modified, a sample carrier 7, coverslip edges 17 and darker areas of holding frame clips 16 arediscernible. It is intended to remove the holding frame clips 16 fromthe microscope image 20 to be modified.

To this end, a projection of the microscope image 20 to be modified intothe feature space W is carried out in process P10. That is to say thatthe feature vector (latent code) w1 is established which, when inputinto the generator G, calculates an output image that is identical or asidentical as possible to the microscope image 20 to be modified. Thefeature vector w1 can be established, for example, by starting with apredetermined or randomly chosen feature vector in the space W anditeratively adjusting its feature variables so that a deviation betweenthe image calculated therefrom and the microscope image 20 to bemodified is minimized.

Once the feature vector w1 has been obtained, the feature variablecorresponding to the image property to be changed is modified in processP11. In this example, the feature variable that determines thevisibility of holding frame clips 16 is modified. The feature variablescan be modified manually or automatically. For example, a slider S orsome other selection option can be displayed to a user for a manualmodification. By means of the change in the feature variables, amodified feature vector w2 is generated. The feature vectors w1 and w2can lie in relation to each other as illustrated in FIG. 6 so that thefeature variable b is changed from the value b1 to the value b2.

The modified feature vector w2 is input in process P12 into thegenerative network Gen, which calculates an output image, called themodified microscope image 30 here, from said modified feature vector w2.The modified microscope image 30 differs from the microscope image 20 tobe modified essentially alone in the changed image property. In thepresent example, the dark image areas of the holding frame clips 16 areaccordingly not as visible or not visible at all in the modifiedmicroscope image 30.

The modified microscope image 30 shown is an actual test resultcalculated with a generator G from the shown microscope image 20 to bemodified, the depiction in FIG. 9 merely being supplemented by aconversion into greyscale.

FIG. 10

FIG. 10 shows a structure and a training of a generative model of afurther example embodiment of a method according to the invention.

A generative model forming part of a GAN was described with reference toFIGS. 3-5 . In the example embodiment of FIG. 10 , on the other hand, adecoder D1 of a variational autoencoder VAE is implemented as thegenerative model Gen.

The variational autoencoder VAE comprises an encoder E1 into whichtraining images T comprising microscope images 20A-20G are input in thetraining. The encoder E1 generates from each of the microscope images20A-20G a respective output 45 which, instead of constituting a point ina feature space Z, indicates a distribution in the feature space Z. Anassociated distribution is output for each feature variable of thefeature space, wherein the distributions can each be defined by adistribution mean μ_(a) to μ_(u) and a distribution width σ_(a) toσ_(u). A point/feature vector z randomly selected from thesedistributions is input into the decoder D1 in the training. The featurevector z comprises concrete numerical values for the feature variables ato u.

The decoder D1 calculates, from the feature vector z, a generatedmicroscope image 40 as its intended output. The generated microscopeimage 40 and the associated microscope image 20A are input into a lossfunction L, which measures deviations between these images. Modelparameter values of the variational autoencoder VAE are iterativelyadjusted in the training in order to minimize the loss function L. Uponcompletion of the training, the variational autoencoder VAE is able toreconstruct an input microscope image, i.e. the output of thevariational autoencoder VAE is essentially identical to the input image.

Since the output 45 of the encoder E1 is a distribution, it is learnedthat points lying close together in the feature space Z from which apoint/feature vector is input into the decoder D1 yield similar results.The feature space Z thereby obtains a structure in a manner similar tothe one described in relation to the feature space of the GAN.

Upon completion of the training, a semantics of feature variables isestablished as described in relation to the feature space of the GAN.

The encoder E1 and the decoder D1 can subsequently be implemented inorder to process a microscope image to be modified in a desired manner.This occurs as described with reference to FIG. 9 . The process P10, inwhich a feature vector is calculated from the microscope image to bemodified, is performed by means of the encoder E1, whose output 45defines the feature vector. For example, it is possible to use thedistribution means μ_(a) to μ_(u) as feature variables a to u of thefeature vector for the input microscope image to be modified. Thedecoder D1 acts as a generative model Gen.

The various processes of methods according to the invention aresummarized with reference to the following figure.

FIG. 11

FIG. 11 shows a flowchart with processes P1 and P2 which must be carriedout before microscope images can be modified with respect to an imageproperty in a targeted manner.

In process P1, a generative model is trained. This can occur asdescribed with reference to FIGS. 3-5 or FIG. 10 .

In process P2, a respective semantics is determined for a plurality offeature variables of a feature space, i.e. it is established which imageproperty is primarily affected by which feature variable. This can occuras described with reference to FIG. 6 .

In process P9, an overview image is captured with a microscope, whichoverview image serves as the microscope image to be modified.

In order to change an image property of the microscope image to bemodified, a projection of the microscope image to be modified into afeature space (latent space) Z or W is calculated in process P10,whereby a feature vector (latent code) z or w is calculated whichrepresents the microscope image to be modified. In process P11, one ormore feature variables of the feature vector z or w are modifiedaccording to the desired change in an image property. Then, in processP12, an image (the modified microscope image) is calculated from themodified feature vector, as recounted with reference to FIG. 9 .

In process P13, the modified microscope image is used in a workflow ofthe microscopy system. For example, the modified microscope image canserve as a navigation map in which a location to be positioned by thesample stage is selected by a user or automatically established by meansof software. The modified microscope image can also be input into asubsequent image processing program, e.g. in order to localize and/oridentify objects in the image.

The described example embodiments are purely illustrative and variantsof the same are possible within the scope of the attached claims.

LIST OF REFERENCE SIGNS

-   1 Microscope-   2 Stand-   3 Objective revolver-   4 (Microscope) objective-   5 Illumination device-   6 Sample stage-   7 Sample carrier-   8 Microscope camera-   9 Overview camera-   9A Field of view of the overview camera-   9B Mirror-   10 Computing device-   11 Computer program-   16 Holding frame-   17 Cover slip edge-   18 Labelling field-   20 Microscope image/microscope image to be modified-   20A-20G Microscope images of the training dataset-   30 Modified microscope image-   40 Generated microscope images-   45 Output of the encoder E1 of the autoencoder VAE-   100 Microscopy system-   a, b, . . . , u Feature variables spanning the feature space-   A Autoencoder-   B Image properties defined by feature variables in the feature space-   d Output/Discrimination result of the discriminator of the GAN-   D Discriminator of the GAN-   D1 Decoder of the autoencoder VAE-   E1 Encoder of the autoencoder VAE-   F Feature variable represented as a combination of the feature    variables a, b, . . . , u-   F1-F3 Components/factors of the feature variables F-   GAN Generative adversarial networks-   G Generator of the GAN-   Gen Generative model, e.g. generator of the GAN or decoder of the    VAE-   L Loss function-   MN Mapping network-   P1 Training a generative model-   P2 Determining a respective semantics of feature variables of a    feature space Z or W-   P9 Capturing an overview image-   P10 Establishing a feature vector z or w for a microscope image to    be modified-   P11 Changing the feature vector z or w in accordance with intended    changes in image properties-   P12 Calculating a modified microscope image from the changed feature    vector by means of the generative model-   P13 Using the modified microscope image in a workflow of the    microscopy system-   S Slider for changing the value of a feature variable-   T Training dataset/training images-   VAE Variational autoencoder-   w Feature vector with feature variables a, b, . . . , u-   w1 Feature vector of the microscope image to be modified with    feature variables a1, b1, . . . , u1-   w2 Modified feature vector with feature variables a1, b2, . . . , u1-   W Feature space spanned by the feature variables a, b, . . . , u-   z (Random) feature vector in the feature space Z-   Z Feature space-   μ_(a)-μ_(u) Distribution means of an output of the encoder of the    VAE-   σ_(a)-σ_(u) Distribution widths of an output of the encoder of the    VAE

What is claimed is:
 1. A computer-implemented method for modifyingmicroscope images, comprising: training a generative model using atraining dataset which comprises a plurality of microscope images,wherein after the training the generative model is configured tocalculate a generated microscope image from a feature vector derivedfrom a feature space; establishing which image properties are affectedby which feature variables in the feature space; receiving a microscopeimage to be modified; projecting the microscope image to be modifiedinto the feature space in order to obtain an associated feature vector;modifying one or more feature variables of the feature vector in orderto change one or more image properties, whereby a modified featurevector is generated; and projecting the modified feature vector backinto an image space by inputting the modified feature vector into thegenerative model, thereby generating a modified microscope image.
 2. Thecomputer-implemented method according to claim 1, wherein the generativemodel is formed by a generator of a generative adversarial network,wherein the modified microscope image is input into a discriminator ofthe generative adversarial network, wherein it is inferred as a functionof an output of the discriminator whether image artefacts were caused bythe modification of the feature vector.
 3. The computer-implementedmethod according to claim 1, wherein the generative model is formed by agenerator of generative adversarial networks or by a decoder of anautoencoder.
 4. The computer-implemented method according to claim 1,wherein the one or more image properties relate to one or more of thefollowing: an exposure of an image or image area; a contamination of asample carrier or cover slip area; a contrast of cover slip edges;reflections, local dimming or other artefacts on a sample carrier;background artefacts visible through a transparent sample carrier; and abackground illumination.
 5. The computer-implemented method according toclaim 1, further comprising: providing a selection option with which auser can specify an intended change in the at least one image property,wherein one or more feature variables of the feature vector are modifiedin accordance with the intended change.
 6. The computer-implementedmethod according to claim 1, wherein one or more of the featurevariables of the feature vector are automatically modified in accordancewith predetermined criteria.
 7. The computer-implemented methodaccording to claim 1, further comprising: inputting the modifiedmicroscope image into a trained inspection model trained to establishwhether image artefacts were caused by the modification of the featurevector.
 8. The computer-implemented method according to claim 1, whereinan image processing program, which is a segmentation model, detectionmodel, classification model or a model for image-to-image mapping,calculates an image processing result for an input image by first:carrying out the method according to claim 1, wherein the modificationof one or more feature variables of the feature vector occurs inaccordance with requirements of the image processing program, andsubsequently: inputting the modified microscope image into the imageprocessing program, which calculates the image processing resulttherefrom.
 9. A computer-implemented method for modifying microscopeimages, comprising: receiving a microscope image to be modified;generating a modified microscope image using a generative model, whereinthe generative model has been trained, using a training dataset whichcomprises a plurality of microscope images, to calculate a generatedmicroscope image from a feature vector derived from a feature space,wherein a relationship between one or more image properties and arespective feature variable in the feature space is given, wherein atleast the following steps are performed to generate the modifiedmicroscope image using the generative model: projecting the microscopeimage to be modified into the feature space in order to obtain anassociated feature vector; modifying one or more feature variables ofthe feature vector in order to change one or more image properties,whereby a modified feature vector is generated; and projecting themodified feature vector back into an image space by inputting themodified feature vector into the generative model, thereby generatingthe modified microscope image.
 10. The computer-implemented methodaccording to claim 9, wherein the generative model is formed by agenerator of generative adversarial networks or by a decoder of anautoencoder.
 11. The computer-implemented method according to claim 9,wherein the one or more image properties relate to one or more of thefollowing: an exposure of an image or image area; a contamination of asample carrier or cover slip area; a contrast of cover slip edges;reflections, local dimming or other artefacts on a sample carrier;background artefacts visible through a transparent sample carrier; and abackground illumination.
 12. The computer-implemented method accordingto claim 9, further comprising: providing a selection option with whicha user can specify an intended change in the at least one imageproperty, wherein one or more feature variables of the feature vectorare modified in accordance with the intended change.
 13. Thecomputer-implemented method according to claim 9, wherein one or more ofthe feature variables of the feature vector are automatically modifiedin accordance with predetermined criteria.
 14. The computer-implementedmethod according to claim 9, further comprising: inputting the modifiedmicroscope image into a trained inspection model trained to establishwhether image artefacts were caused by the modification of the featurevector.
 15. The computer-implemented method according to claim 9,wherein the generative model is formed by a generator of a generativeadversarial network, wherein the modified microscope image is input intoa discriminator of the generative adversarial network, wherein it isinferred as a function of an output of the discriminator whether imageartefacts were caused by the modification of the feature vector.
 16. Thecomputer-implemented method according to claim 9, wherein an imageprocessing program, which is a segmentation model, detection model,classification model or a model for image-to-image mapping, calculatesan image processing result for an input image by first: carrying out themethod according to claim 9, wherein the modification of one or morefeature variables of the feature vector occurs in accordance withrequirements of the image processing program, and subsequently:inputting the modified microscope image into the image processingprogram, which calculates the image processing result therefrom.
 17. Amicroscopy system, comprising: a microscope for capturing microscopeimages; and a computing device which is configured to execute the methodaccording to claim
 1. 18. A computer-readable storage medium, comprisingcommands which, when executed by a computer, cause the computer toexecute the method according to claim 1.